{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Siraj's 03 Challenge\n",
    "\n",
    "### This is a response to the Coding Challenge in: [https://youtu.be/si8zZHkufRY](https://youtu.be/si8zZHkufRY?t=9m4s)\n",
    "\n",
    "> The challenge for this video is to train a model on this dataset of video game reviews from IGN.com. Then, given some new video game title it should be able to classify it. You can use pandas to parse this dataset. Right now each review has a label that's either Amazing, Great, Good, Mediocre, Painful, or Awful. These are the emotions. Using the existing labels is extra credit. The baseline is that you can just convert the labels so that there are only 2 emotions (positive or negative). Ideally you can use an RNN via TFLearn like the one in this example, but I'll accept other types of ML models as well. You'll learn how to parse data, select appropriate features, and use a neural net on an IRL problem. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Sentiment Labels to be Predicted\n",
    "\n",
    "- Great          \n",
    "- Good           \n",
    "- Okay           \n",
    "- Mediocre       \n",
    "- Amazing        \n",
    "- Bad            \n",
    "- Awful          \n",
    "- Painful        \n",
    "- Unbearable     \n",
    "- Masterpiece    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Accuracy Results \n",
    "- `Dummy Classifier (i.e. select most frequent class): 0.25631 (25.6%)`\n",
    "- `Multinomial Naive Bayes:                            0.32355 (32.4%)`\n",
    "- `RNN (using tflearn):                                0.41546 (41.5%)`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34mPython Version: 2.7.13\u001b[0m\n",
      "\u001b[35mTensorFlow Ver: 1.3.0\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "import sys\n",
    "import tensorflow as tf\n",
    "from termcolor import colored\n",
    "print(colored('Python Version: %s' % sys.version.split()[0], 'blue'))\n",
    "print(colored('TensorFlow Ver: %s' % tf.__version__, 'magenta'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Enter no. of epochs for RNN training: 100\n"
     ]
    }
   ],
   "source": [
    "n_epoch = int(input('Enter no. of epochs for RNN training: '))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[31mNo. of epochs: 100\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "print(colored('No. of epochs: %d' % n_epoch, 'red'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "pd.set_option('display.max_colwidth', 1000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Load IGN Dataset as `original_ign`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>score_phrase</th>\n",
       "      <th>title</th>\n",
       "      <th>url</th>\n",
       "      <th>platform</th>\n",
       "      <th>score</th>\n",
       "      <th>genre</th>\n",
       "      <th>editors_choice</th>\n",
       "      <th>release_year</th>\n",
       "      <th>release_month</th>\n",
       "      <th>release_day</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>Amazing</td>\n",
       "      <td>LittleBigPlanet PS Vita</td>\n",
       "      <td>/games/littlebigplanet-vita/vita-98907</td>\n",
       "      <td>PlayStation Vita</td>\n",
       "      <td>9.0</td>\n",
       "      <td>Platformer</td>\n",
       "      <td>Y</td>\n",
       "      <td>2012</td>\n",
       "      <td>9</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>Amazing</td>\n",
       "      <td>LittleBigPlanet PS Vita -- Marvel Super Hero Edition</td>\n",
       "      <td>/games/littlebigplanet-ps-vita-marvel-super-hero-edition/vita-20027059</td>\n",
       "      <td>PlayStation Vita</td>\n",
       "      <td>9.0</td>\n",
       "      <td>Platformer</td>\n",
       "      <td>Y</td>\n",
       "      <td>2012</td>\n",
       "      <td>9</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>Great</td>\n",
       "      <td>Splice: Tree of Life</td>\n",
       "      <td>/games/splice/ipad-141070</td>\n",
       "      <td>iPad</td>\n",
       "      <td>8.5</td>\n",
       "      <td>Puzzle</td>\n",
       "      <td>N</td>\n",
       "      <td>2012</td>\n",
       "      <td>9</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>Great</td>\n",
       "      <td>NHL 13</td>\n",
       "      <td>/games/nhl-13/xbox-360-128182</td>\n",
       "      <td>Xbox 360</td>\n",
       "      <td>8.5</td>\n",
       "      <td>Sports</td>\n",
       "      <td>N</td>\n",
       "      <td>2012</td>\n",
       "      <td>9</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>Great</td>\n",
       "      <td>NHL 13</td>\n",
       "      <td>/games/nhl-13/ps3-128181</td>\n",
       "      <td>PlayStation 3</td>\n",
       "      <td>8.5</td>\n",
       "      <td>Sports</td>\n",
       "      <td>N</td>\n",
       "      <td>2012</td>\n",
       "      <td>9</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>5</td>\n",
       "      <td>Good</td>\n",
       "      <td>Total War Battles: Shogun</td>\n",
       "      <td>/games/total-war-battles-shogun/mac-142565</td>\n",
       "      <td>Macintosh</td>\n",
       "      <td>7.0</td>\n",
       "      <td>Strategy</td>\n",
       "      <td>N</td>\n",
       "      <td>2012</td>\n",
       "      <td>9</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>6</td>\n",
       "      <td>Awful</td>\n",
       "      <td>Double Dragon: Neon</td>\n",
       "      <td>/games/double-dragon-neon/xbox-360-131320</td>\n",
       "      <td>Xbox 360</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Fighting</td>\n",
       "      <td>N</td>\n",
       "      <td>2012</td>\n",
       "      <td>9</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>7</td>\n",
       "      <td>Amazing</td>\n",
       "      <td>Guild Wars 2</td>\n",
       "      <td>/games/guild-wars-2/pc-896298</td>\n",
       "      <td>PC</td>\n",
       "      <td>9.0</td>\n",
       "      <td>RPG</td>\n",
       "      <td>Y</td>\n",
       "      <td>2012</td>\n",
       "      <td>9</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>8</td>\n",
       "      <td>Awful</td>\n",
       "      <td>Double Dragon: Neon</td>\n",
       "      <td>/games/double-dragon-neon/ps3-131321</td>\n",
       "      <td>PlayStation 3</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Fighting</td>\n",
       "      <td>N</td>\n",
       "      <td>2012</td>\n",
       "      <td>9</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>9</td>\n",
       "      <td>Good</td>\n",
       "      <td>Total War Battles: Shogun</td>\n",
       "      <td>/games/total-war-battles-shogun/pc-142564</td>\n",
       "      <td>PC</td>\n",
       "      <td>7.0</td>\n",
       "      <td>Strategy</td>\n",
       "      <td>N</td>\n",
       "      <td>2012</td>\n",
       "      <td>9</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0 score_phrase  \\\n",
       "0           0      Amazing   \n",
       "1           1      Amazing   \n",
       "2           2        Great   \n",
       "3           3        Great   \n",
       "4           4        Great   \n",
       "5           5         Good   \n",
       "6           6        Awful   \n",
       "7           7      Amazing   \n",
       "8           8        Awful   \n",
       "9           9         Good   \n",
       "\n",
       "                                                  title  \\\n",
       "0                               LittleBigPlanet PS Vita   \n",
       "1  LittleBigPlanet PS Vita -- Marvel Super Hero Edition   \n",
       "2                                  Splice: Tree of Life   \n",
       "3                                                NHL 13   \n",
       "4                                                NHL 13   \n",
       "5                             Total War Battles: Shogun   \n",
       "6                                   Double Dragon: Neon   \n",
       "7                                          Guild Wars 2   \n",
       "8                                   Double Dragon: Neon   \n",
       "9                             Total War Battles: Shogun   \n",
       "\n",
       "                                                                      url  \\\n",
       "0                                  /games/littlebigplanet-vita/vita-98907   \n",
       "1  /games/littlebigplanet-ps-vita-marvel-super-hero-edition/vita-20027059   \n",
       "2                                               /games/splice/ipad-141070   \n",
       "3                                           /games/nhl-13/xbox-360-128182   \n",
       "4                                                /games/nhl-13/ps3-128181   \n",
       "5                              /games/total-war-battles-shogun/mac-142565   \n",
       "6                               /games/double-dragon-neon/xbox-360-131320   \n",
       "7                                           /games/guild-wars-2/pc-896298   \n",
       "8                                    /games/double-dragon-neon/ps3-131321   \n",
       "9                               /games/total-war-battles-shogun/pc-142564   \n",
       "\n",
       "           platform  score       genre editors_choice  release_year  \\\n",
       "0  PlayStation Vita    9.0  Platformer              Y          2012   \n",
       "1  PlayStation Vita    9.0  Platformer              Y          2012   \n",
       "2              iPad    8.5      Puzzle              N          2012   \n",
       "3          Xbox 360    8.5      Sports              N          2012   \n",
       "4     PlayStation 3    8.5      Sports              N          2012   \n",
       "5         Macintosh    7.0    Strategy              N          2012   \n",
       "6          Xbox 360    3.0    Fighting              N          2012   \n",
       "7                PC    9.0         RPG              Y          2012   \n",
       "8     PlayStation 3    3.0    Fighting              N          2012   \n",
       "9                PC    7.0    Strategy              N          2012   \n",
       "\n",
       "   release_month  release_day  \n",
       "0              9           12  \n",
       "1              9           12  \n",
       "2              9           12  \n",
       "3              9           11  \n",
       "4              9           11  \n",
       "5              9           11  \n",
       "6              9           11  \n",
       "7              9           11  \n",
       "8              9           11  \n",
       "9              9           11  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "original_ign = pd.read_csv('input/ign.csv')\n",
    "original_ign.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Check out the `shape` of the IGN Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('original_ign.shape:', (18625, 11))\n"
     ]
    }
   ],
   "source": [
    "print('original_ign.shape:', original_ign.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Check out all the unique `score_phrase` as well as their `counts`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Great          4773\n",
       "Good           4741\n",
       "Okay           2945\n",
       "Mediocre       1959\n",
       "Amazing        1804\n",
       "Bad            1269\n",
       "Awful           664\n",
       "Painful         340\n",
       "Unbearable       72\n",
       "Masterpiece      55\n",
       "Disaster          3\n",
       "Name: score_phrase, dtype: int64"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "original_ign.score_phrase.value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Preprocessing\n",
    "\n",
    "As always, we gotta perform preprocessing on our Dataset before training our model(s)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Convert `score_phrase` to binary sentiments and add a new column called `sentiment`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "bad_phrases = ['Bad', 'Awful', 'Painful', 'Unbearable', 'Disaster']\n",
    "original_ign['sentiment'] = original_ign.score_phrase.isin(bad_phrases).map({True: 'Negative', False: 'Positive'})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Remove \"Disaster\"\n",
    "original_ign = original_ign[original_ign['score_phrase'] != 'Disaster']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>score_phrase</th>\n",
       "      <th>title</th>\n",
       "      <th>url</th>\n",
       "      <th>platform</th>\n",
       "      <th>score</th>\n",
       "      <th>genre</th>\n",
       "      <th>editors_choice</th>\n",
       "      <th>release_year</th>\n",
       "      <th>release_month</th>\n",
       "      <th>release_day</th>\n",
       "      <th>sentiment</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>Amazing</td>\n",
       "      <td>LittleBigPlanet PS Vita</td>\n",
       "      <td>/games/littlebigplanet-vita/vita-98907</td>\n",
       "      <td>PlayStation Vita</td>\n",
       "      <td>9.0</td>\n",
       "      <td>Platformer</td>\n",
       "      <td>Y</td>\n",
       "      <td>2012</td>\n",
       "      <td>9</td>\n",
       "      <td>12</td>\n",
       "      <td>Positive</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>Amazing</td>\n",
       "      <td>LittleBigPlanet PS Vita -- Marvel Super Hero Edition</td>\n",
       "      <td>/games/littlebigplanet-ps-vita-marvel-super-hero-edition/vita-20027059</td>\n",
       "      <td>PlayStation Vita</td>\n",
       "      <td>9.0</td>\n",
       "      <td>Platformer</td>\n",
       "      <td>Y</td>\n",
       "      <td>2012</td>\n",
       "      <td>9</td>\n",
       "      <td>12</td>\n",
       "      <td>Positive</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>Great</td>\n",
       "      <td>Splice: Tree of Life</td>\n",
       "      <td>/games/splice/ipad-141070</td>\n",
       "      <td>iPad</td>\n",
       "      <td>8.5</td>\n",
       "      <td>Puzzle</td>\n",
       "      <td>N</td>\n",
       "      <td>2012</td>\n",
       "      <td>9</td>\n",
       "      <td>12</td>\n",
       "      <td>Positive</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>Great</td>\n",
       "      <td>NHL 13</td>\n",
       "      <td>/games/nhl-13/xbox-360-128182</td>\n",
       "      <td>Xbox 360</td>\n",
       "      <td>8.5</td>\n",
       "      <td>Sports</td>\n",
       "      <td>N</td>\n",
       "      <td>2012</td>\n",
       "      <td>9</td>\n",
       "      <td>11</td>\n",
       "      <td>Positive</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>Great</td>\n",
       "      <td>NHL 13</td>\n",
       "      <td>/games/nhl-13/ps3-128181</td>\n",
       "      <td>PlayStation 3</td>\n",
       "      <td>8.5</td>\n",
       "      <td>Sports</td>\n",
       "      <td>N</td>\n",
       "      <td>2012</td>\n",
       "      <td>9</td>\n",
       "      <td>11</td>\n",
       "      <td>Positive</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0 score_phrase  \\\n",
       "0           0      Amazing   \n",
       "1           1      Amazing   \n",
       "2           2        Great   \n",
       "3           3        Great   \n",
       "4           4        Great   \n",
       "\n",
       "                                                  title  \\\n",
       "0                               LittleBigPlanet PS Vita   \n",
       "1  LittleBigPlanet PS Vita -- Marvel Super Hero Edition   \n",
       "2                                  Splice: Tree of Life   \n",
       "3                                                NHL 13   \n",
       "4                                                NHL 13   \n",
       "\n",
       "                                                                      url  \\\n",
       "0                                  /games/littlebigplanet-vita/vita-98907   \n",
       "1  /games/littlebigplanet-ps-vita-marvel-super-hero-edition/vita-20027059   \n",
       "2                                               /games/splice/ipad-141070   \n",
       "3                                           /games/nhl-13/xbox-360-128182   \n",
       "4                                                /games/nhl-13/ps3-128181   \n",
       "\n",
       "           platform  score       genre editors_choice  release_year  \\\n",
       "0  PlayStation Vita    9.0  Platformer              Y          2012   \n",
       "1  PlayStation Vita    9.0  Platformer              Y          2012   \n",
       "2              iPad    8.5      Puzzle              N          2012   \n",
       "3          Xbox 360    8.5      Sports              N          2012   \n",
       "4     PlayStation 3    8.5      Sports              N          2012   \n",
       "\n",
       "   release_month  release_day sentiment  \n",
       "0              9           12  Positive  \n",
       "1              9           12  Positive  \n",
       "2              9           12  Positive  \n",
       "3              9           11  Positive  \n",
       "4              9           11  Positive  "
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "original_ign.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### No. of Positive Sentiments VS No. of Negative Seniments"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Positive    0.874074\n",
       "Negative    0.125926\n",
       "Name: sentiment, dtype: float64"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "original_ign.sentiment.value_counts(normalize=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Check for null elements"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Unnamed: 0         0\n",
       "score_phrase       0\n",
       "title              0\n",
       "url                0\n",
       "platform           0\n",
       "score              0\n",
       "genre             36\n",
       "editors_choice     0\n",
       "release_year       0\n",
       "release_month      0\n",
       "release_day        0\n",
       "sentiment          0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "original_ign.isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Fill all null elements with an empty string"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "original_ign.fillna(value='', inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# original_ign[ original_ign['genre'] == '' ].shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Create a new DataFrame called `ign`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sentiment</th>\n",
       "      <th>score_phrase</th>\n",
       "      <th>title</th>\n",
       "      <th>platform</th>\n",
       "      <th>genre</th>\n",
       "      <th>editors_choice</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Positive</td>\n",
       "      <td>Amazing</td>\n",
       "      <td>LittleBigPlanet PS Vita</td>\n",
       "      <td>PlayStation Vita</td>\n",
       "      <td>Platformer</td>\n",
       "      <td>Y</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Positive</td>\n",
       "      <td>Amazing</td>\n",
       "      <td>LittleBigPlanet PS Vita -- Marvel Super Hero Edition</td>\n",
       "      <td>PlayStation Vita</td>\n",
       "      <td>Platformer</td>\n",
       "      <td>Y</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Positive</td>\n",
       "      <td>Great</td>\n",
       "      <td>Splice: Tree of Life</td>\n",
       "      <td>iPad</td>\n",
       "      <td>Puzzle</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Positive</td>\n",
       "      <td>Great</td>\n",
       "      <td>NHL 13</td>\n",
       "      <td>Xbox 360</td>\n",
       "      <td>Sports</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Positive</td>\n",
       "      <td>Great</td>\n",
       "      <td>NHL 13</td>\n",
       "      <td>PlayStation 3</td>\n",
       "      <td>Sports</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Positive</td>\n",
       "      <td>Good</td>\n",
       "      <td>Total War Battles: Shogun</td>\n",
       "      <td>Macintosh</td>\n",
       "      <td>Strategy</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Negative</td>\n",
       "      <td>Awful</td>\n",
       "      <td>Double Dragon: Neon</td>\n",
       "      <td>Xbox 360</td>\n",
       "      <td>Fighting</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Positive</td>\n",
       "      <td>Amazing</td>\n",
       "      <td>Guild Wars 2</td>\n",
       "      <td>PC</td>\n",
       "      <td>RPG</td>\n",
       "      <td>Y</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Negative</td>\n",
       "      <td>Awful</td>\n",
       "      <td>Double Dragon: Neon</td>\n",
       "      <td>PlayStation 3</td>\n",
       "      <td>Fighting</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Positive</td>\n",
       "      <td>Good</td>\n",
       "      <td>Total War Battles: Shogun</td>\n",
       "      <td>PC</td>\n",
       "      <td>Strategy</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  sentiment score_phrase  \\\n",
       "0  Positive      Amazing   \n",
       "1  Positive      Amazing   \n",
       "2  Positive        Great   \n",
       "3  Positive        Great   \n",
       "4  Positive        Great   \n",
       "5  Positive         Good   \n",
       "6  Negative        Awful   \n",
       "7  Positive      Amazing   \n",
       "8  Negative        Awful   \n",
       "9  Positive         Good   \n",
       "\n",
       "                                                  title          platform  \\\n",
       "0                               LittleBigPlanet PS Vita  PlayStation Vita   \n",
       "1  LittleBigPlanet PS Vita -- Marvel Super Hero Edition  PlayStation Vita   \n",
       "2                                  Splice: Tree of Life              iPad   \n",
       "3                                                NHL 13          Xbox 360   \n",
       "4                                                NHL 13     PlayStation 3   \n",
       "5                             Total War Battles: Shogun         Macintosh   \n",
       "6                                   Double Dragon: Neon          Xbox 360   \n",
       "7                                          Guild Wars 2                PC   \n",
       "8                                   Double Dragon: Neon     PlayStation 3   \n",
       "9                             Total War Battles: Shogun                PC   \n",
       "\n",
       "        genre editors_choice  \n",
       "0  Platformer              Y  \n",
       "1  Platformer              Y  \n",
       "2      Puzzle              N  \n",
       "3      Sports              N  \n",
       "4      Sports              N  \n",
       "5    Strategy              N  \n",
       "6    Fighting              N  \n",
       "7         RPG              Y  \n",
       "8    Fighting              N  \n",
       "9    Strategy              N  "
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ign = original_ign[ ['sentiment', 'score_phrase', 'title', 'platform', 'genre', 'editors_choice'] ].copy()\n",
    "ign.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create a new column called `is_editors_choice`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sentiment</th>\n",
       "      <th>score_phrase</th>\n",
       "      <th>title</th>\n",
       "      <th>platform</th>\n",
       "      <th>genre</th>\n",
       "      <th>editors_choice</th>\n",
       "      <th>is_editors_choice</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Positive</td>\n",
       "      <td>Amazing</td>\n",
       "      <td>LittleBigPlanet PS Vita</td>\n",
       "      <td>PlayStation Vita</td>\n",
       "      <td>Platformer</td>\n",
       "      <td>Y</td>\n",
       "      <td>editors_choice</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Positive</td>\n",
       "      <td>Amazing</td>\n",
       "      <td>LittleBigPlanet PS Vita -- Marvel Super Hero Edition</td>\n",
       "      <td>PlayStation Vita</td>\n",
       "      <td>Platformer</td>\n",
       "      <td>Y</td>\n",
       "      <td>editors_choice</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Positive</td>\n",
       "      <td>Great</td>\n",
       "      <td>Splice: Tree of Life</td>\n",
       "      <td>iPad</td>\n",
       "      <td>Puzzle</td>\n",
       "      <td>N</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Positive</td>\n",
       "      <td>Great</td>\n",
       "      <td>NHL 13</td>\n",
       "      <td>Xbox 360</td>\n",
       "      <td>Sports</td>\n",
       "      <td>N</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Positive</td>\n",
       "      <td>Great</td>\n",
       "      <td>NHL 13</td>\n",
       "      <td>PlayStation 3</td>\n",
       "      <td>Sports</td>\n",
       "      <td>N</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  sentiment score_phrase  \\\n",
       "0  Positive      Amazing   \n",
       "1  Positive      Amazing   \n",
       "2  Positive        Great   \n",
       "3  Positive        Great   \n",
       "4  Positive        Great   \n",
       "\n",
       "                                                  title          platform  \\\n",
       "0                               LittleBigPlanet PS Vita  PlayStation Vita   \n",
       "1  LittleBigPlanet PS Vita -- Marvel Super Hero Edition  PlayStation Vita   \n",
       "2                                  Splice: Tree of Life              iPad   \n",
       "3                                                NHL 13          Xbox 360   \n",
       "4                                                NHL 13     PlayStation 3   \n",
       "\n",
       "        genre editors_choice is_editors_choice  \n",
       "0  Platformer              Y    editors_choice  \n",
       "1  Platformer              Y    editors_choice  \n",
       "2      Puzzle              N                    \n",
       "3      Sports              N                    \n",
       "4      Sports              N                    "
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ign['is_editors_choice'] = ign['editors_choice'].map({'Y': 'editors_choice', 'N': ''})\n",
    "ign.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create a new column called `text` which contains contents of several columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "ign['text'] = ign['title'].str.cat(ign['platform'], sep=' ').str.cat(ign['genre'], sep=' ').str.cat(ign['is_editors_choice'], sep=' ')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('Shape of \"ign\" DataFrame:', (18622, 8))\n"
     ]
    }
   ],
   "source": [
    "print('Shape of \\\"ign\\\" DataFrame:', ign.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sentiment</th>\n",
       "      <th>score_phrase</th>\n",
       "      <th>title</th>\n",
       "      <th>platform</th>\n",
       "      <th>genre</th>\n",
       "      <th>editors_choice</th>\n",
       "      <th>is_editors_choice</th>\n",
       "      <th>text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Positive</td>\n",
       "      <td>Amazing</td>\n",
       "      <td>LittleBigPlanet PS Vita</td>\n",
       "      <td>PlayStation Vita</td>\n",
       "      <td>Platformer</td>\n",
       "      <td>Y</td>\n",
       "      <td>editors_choice</td>\n",
       "      <td>LittleBigPlanet PS Vita PlayStation Vita Platformer editors_choice</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Positive</td>\n",
       "      <td>Amazing</td>\n",
       "      <td>LittleBigPlanet PS Vita -- Marvel Super Hero Edition</td>\n",
       "      <td>PlayStation Vita</td>\n",
       "      <td>Platformer</td>\n",
       "      <td>Y</td>\n",
       "      <td>editors_choice</td>\n",
       "      <td>LittleBigPlanet PS Vita -- Marvel Super Hero Edition PlayStation Vita Platformer editors_choice</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Positive</td>\n",
       "      <td>Great</td>\n",
       "      <td>Splice: Tree of Life</td>\n",
       "      <td>iPad</td>\n",
       "      <td>Puzzle</td>\n",
       "      <td>N</td>\n",
       "      <td></td>\n",
       "      <td>Splice: Tree of Life iPad Puzzle</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Positive</td>\n",
       "      <td>Great</td>\n",
       "      <td>NHL 13</td>\n",
       "      <td>Xbox 360</td>\n",
       "      <td>Sports</td>\n",
       "      <td>N</td>\n",
       "      <td></td>\n",
       "      <td>NHL 13 Xbox 360 Sports</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Positive</td>\n",
       "      <td>Great</td>\n",
       "      <td>NHL 13</td>\n",
       "      <td>PlayStation 3</td>\n",
       "      <td>Sports</td>\n",
       "      <td>N</td>\n",
       "      <td></td>\n",
       "      <td>NHL 13 PlayStation 3 Sports</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Positive</td>\n",
       "      <td>Good</td>\n",
       "      <td>Total War Battles: Shogun</td>\n",
       "      <td>Macintosh</td>\n",
       "      <td>Strategy</td>\n",
       "      <td>N</td>\n",
       "      <td></td>\n",
       "      <td>Total War Battles: Shogun Macintosh Strategy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Negative</td>\n",
       "      <td>Awful</td>\n",
       "      <td>Double Dragon: Neon</td>\n",
       "      <td>Xbox 360</td>\n",
       "      <td>Fighting</td>\n",
       "      <td>N</td>\n",
       "      <td></td>\n",
       "      <td>Double Dragon: Neon Xbox 360 Fighting</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Positive</td>\n",
       "      <td>Amazing</td>\n",
       "      <td>Guild Wars 2</td>\n",
       "      <td>PC</td>\n",
       "      <td>RPG</td>\n",
       "      <td>Y</td>\n",
       "      <td>editors_choice</td>\n",
       "      <td>Guild Wars 2 PC RPG editors_choice</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Negative</td>\n",
       "      <td>Awful</td>\n",
       "      <td>Double Dragon: Neon</td>\n",
       "      <td>PlayStation 3</td>\n",
       "      <td>Fighting</td>\n",
       "      <td>N</td>\n",
       "      <td></td>\n",
       "      <td>Double Dragon: Neon PlayStation 3 Fighting</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Positive</td>\n",
       "      <td>Good</td>\n",
       "      <td>Total War Battles: Shogun</td>\n",
       "      <td>PC</td>\n",
       "      <td>Strategy</td>\n",
       "      <td>N</td>\n",
       "      <td></td>\n",
       "      <td>Total War Battles: Shogun PC Strategy</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  sentiment score_phrase  \\\n",
       "0  Positive      Amazing   \n",
       "1  Positive      Amazing   \n",
       "2  Positive        Great   \n",
       "3  Positive        Great   \n",
       "4  Positive        Great   \n",
       "5  Positive         Good   \n",
       "6  Negative        Awful   \n",
       "7  Positive      Amazing   \n",
       "8  Negative        Awful   \n",
       "9  Positive         Good   \n",
       "\n",
       "                                                  title          platform  \\\n",
       "0                               LittleBigPlanet PS Vita  PlayStation Vita   \n",
       "1  LittleBigPlanet PS Vita -- Marvel Super Hero Edition  PlayStation Vita   \n",
       "2                                  Splice: Tree of Life              iPad   \n",
       "3                                                NHL 13          Xbox 360   \n",
       "4                                                NHL 13     PlayStation 3   \n",
       "5                             Total War Battles: Shogun         Macintosh   \n",
       "6                                   Double Dragon: Neon          Xbox 360   \n",
       "7                                          Guild Wars 2                PC   \n",
       "8                                   Double Dragon: Neon     PlayStation 3   \n",
       "9                             Total War Battles: Shogun                PC   \n",
       "\n",
       "        genre editors_choice is_editors_choice  \\\n",
       "0  Platformer              Y    editors_choice   \n",
       "1  Platformer              Y    editors_choice   \n",
       "2      Puzzle              N                     \n",
       "3      Sports              N                     \n",
       "4      Sports              N                     \n",
       "5    Strategy              N                     \n",
       "6    Fighting              N                     \n",
       "7         RPG              Y    editors_choice   \n",
       "8    Fighting              N                     \n",
       "9    Strategy              N                     \n",
       "\n",
       "                                                                                              text  \n",
       "0                               LittleBigPlanet PS Vita PlayStation Vita Platformer editors_choice  \n",
       "1  LittleBigPlanet PS Vita -- Marvel Super Hero Edition PlayStation Vita Platformer editors_choice  \n",
       "2                                                                Splice: Tree of Life iPad Puzzle   \n",
       "3                                                                          NHL 13 Xbox 360 Sports   \n",
       "4                                                                     NHL 13 PlayStation 3 Sports   \n",
       "5                                                    Total War Battles: Shogun Macintosh Strategy   \n",
       "6                                                           Double Dragon: Neon Xbox 360 Fighting   \n",
       "7                                                               Guild Wars 2 PC RPG editors_choice  \n",
       "8                                                      Double Dragon: Neon PlayStation 3 Fighting   \n",
       "9                                                           Total War Battles: Shogun PC Strategy   "
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ign.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![http://www.westernbands.net/userdata/news_picupload/pic_sid1070-0-norm.jpg](http://www.westernbands.net/userdata/news_picupload/pic_sid1070-0-norm.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Here, I'll treat this as a `multiclass problem` where I attempt to predict the labels (i.e. the `score_phrases`)\n",
    "\n",
    "Examples of **score_phrases**: \n",
    "- Great        \n",
    "- Good         \n",
    "- Okay        \n",
    "- Mediocre    \n",
    "- Amazing      \n",
    "- Bad        \n",
    "- Awful        \n",
    "- Painful      \n",
    "- Unbearable      \n",
    "- Masterpiece     "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X = ign.text\n",
    "y = ign.score_phrase"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Top 10 rows for `X`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0                                 LittleBigPlanet PS Vita PlayStation Vita Platformer editors_choice\n",
       "1    LittleBigPlanet PS Vita -- Marvel Super Hero Edition PlayStation Vita Platformer editors_choice\n",
       "2                                                                  Splice: Tree of Life iPad Puzzle \n",
       "3                                                                            NHL 13 Xbox 360 Sports \n",
       "4                                                                       NHL 13 PlayStation 3 Sports \n",
       "5                                                      Total War Battles: Shogun Macintosh Strategy \n",
       "6                                                             Double Dragon: Neon Xbox 360 Fighting \n",
       "7                                                                 Guild Wars 2 PC RPG editors_choice\n",
       "8                                                        Double Dragon: Neon PlayStation 3 Fighting \n",
       "9                                                             Total War Battles: Shogun PC Strategy \n",
       "Name: text, dtype: object"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Top 10 rows for `y`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    Amazing\n",
       "1    Amazing\n",
       "2      Great\n",
       "3      Great\n",
       "4      Great\n",
       "5       Good\n",
       "6      Awful\n",
       "7    Amazing\n",
       "8      Awful\n",
       "9       Good\n",
       "Name: score_phrase, dtype: object"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model #0: The DUMMY Classifier (Always Choose the *Most Frequent* Class)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "from sklearn.dummy import DummyClassifier\n",
    "from sklearn.pipeline import make_pipeline\n",
    "from sklearn.model_selection import cross_val_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "vect = TfidfVectorizer(stop_words='english', token_pattern=r'\\b\\w{2,}\\b')\n",
    "dummy = DummyClassifier(strategy='most_frequent', random_state=0)\n",
    "\n",
    "dummy_pipeline = make_pipeline(vect, dummy)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'dummyclassifier': DummyClassifier(constant=None, random_state=0, strategy='most_frequent'),\n",
       " 'tfidfvectorizer': TfidfVectorizer(analyzer=u'word', binary=False, decode_error=u'strict',\n",
       "         dtype=<type 'numpy.int64'>, encoding=u'utf-8', input=u'content',\n",
       "         lowercase=True, max_df=1.0, max_features=None, min_df=1,\n",
       "         ngram_range=(1, 1), norm=u'l2', preprocessor=None, smooth_idf=True,\n",
       "         stop_words='english', strip_accents=None, sublinear_tf=False,\n",
       "         token_pattern='\\\\b\\\\w{2,}\\\\b', tokenizer=None, use_idf=True,\n",
       "         vocabulary=None)}"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dummy_pipeline.named_steps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33m\n",
      "Dummy Classifier's Accuracy: 0.25631\n",
      "\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "# Cross Validation\n",
    "cv = cross_val_score(dummy_pipeline, X, y, scoring='accuracy', cv=10, n_jobs=-1)\n",
    "print(colored('\\nDummy Classifier\\'s Accuracy: %0.5f\\n' % cv.mean(), 'yellow'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model #1: MultinomialNB Classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.naive_bayes import MultinomialNB"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "vect = TfidfVectorizer(stop_words='english', \n",
    "                       token_pattern=r'\\b\\w{2,}\\b',\n",
    "                       min_df=1, max_df=0.1,\n",
    "                       ngram_range=(1,2))\n",
    "mnb = MultinomialNB(alpha=2)\n",
    "\n",
    "mnb_pipeline = make_pipeline(vect, mnb)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'multinomialnb': MultinomialNB(alpha=2, class_prior=None, fit_prior=True),\n",
       " 'tfidfvectorizer': TfidfVectorizer(analyzer=u'word', binary=False, decode_error=u'strict',\n",
       "         dtype=<type 'numpy.int64'>, encoding=u'utf-8', input=u'content',\n",
       "         lowercase=True, max_df=0.1, max_features=None, min_df=1,\n",
       "         ngram_range=(1, 2), norm=u'l2', preprocessor=None, smooth_idf=True,\n",
       "         stop_words='english', strip_accents=None, sublinear_tf=False,\n",
       "         token_pattern='\\\\b\\\\w{2,}\\\\b', tokenizer=None, use_idf=True,\n",
       "         vocabulary=None)}"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mnb_pipeline.named_steps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32m\n",
      "MultinomialNB Classifier's Accuracy: 0.32355\n",
      "\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "# Cross Validation\n",
    "cv = cross_val_score(mnb_pipeline, X, y, scoring='accuracy', cv=10, n_jobs=-1)\n",
    "print(colored('\\nMultinomialNB Classifier\\'s Accuracy: %0.5f\\n' % cv.mean(), 'green'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model #2: RNN Classifier using `TFLearn`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import tflearn\n",
    "from tflearn.data_utils import to_categorical, pad_sequences\n",
    "from tflearn.datasets import imdb"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train-Test-Split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.1, random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create the `vocab` (so that we can create `X_word_ids` from `X`)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "vect = CountVectorizer(ngram_range=(1,1), token_pattern=r'\\b\\w{1,}\\b')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "vect.fit(X_train)\n",
    "vocab = vect.vocabulary_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def convert_X_to_X_word_ids(X):\n",
    "    return X.apply( lambda x: [vocab[w] for w in [w.lower().strip() for w in x.split()] if w in vocab] )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X_train_word_ids = convert_X_to_X_word_ids(X_train)\n",
    "X_test_word_ids  = convert_X_to_X_word_ids(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Difference between X(_train/_test) and X(_train_word_ids/test_word_ids)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "16138               Castlevania: Harmony of Despair PlayStation 3 Action, Adventure \n",
       "5945     Kim Possible 2: Drakken's Demise Game Boy Advance Platformer editors_choice\n",
       "11360                                            Madden NFL 09 PlayStation 2 Sports \n",
       "18270                                                   WWE 2K16 Xbox One Wrestling \n",
       "12533                                        The Last Ninja Commodore 64/128 Action \n",
       "Name: text, dtype: object"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "16138                 [3135, 4719, 1912, 5076, 149, 269]\n",
       "5945     [3731, 5128, 1889, 2800, 1026, 266, 5064, 2228]\n",
       "11360                   [4039, 4587, 13, 5076, 77, 6259]\n",
       "18270                      [7450, 136, 7461, 4753, 7442]\n",
       "12533                      [6676, 3844, 4619, 1534, 251]\n",
       "Name: text, dtype: object"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train_word_ids.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('X_train_word_ids.shape:', (16759,))\n",
      "('X_test_word_ids.shape:', (1863,))\n"
     ]
    }
   ],
   "source": [
    "print('X_train_word_ids.shape:', X_train_word_ids.shape)\n",
    "print('X_test_word_ids.shape:', X_test_word_ids.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sequence Padding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X_train_padded_seqs = pad_sequences(X_train_word_ids, maxlen=20, value=0)\n",
    "X_test_padded_seqs  = pad_sequences(X_test_word_ids , maxlen=20, value=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('X_train_padded_seqs.shape:', (16759, 20))\n",
      "('X_test_padded_seqs.shape:', (1863, 20))\n"
     ]
    }
   ],
   "source": [
    "print('X_train_padded_seqs.shape:', X_train_padded_seqs.shape)\n",
    "print('X_test_padded_seqs.shape:', X_test_padded_seqs.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>10</th>\n",
       "      <th>11</th>\n",
       "      <th>12</th>\n",
       "      <th>13</th>\n",
       "      <th>14</th>\n",
       "      <th>15</th>\n",
       "      <th>16</th>\n",
       "      <th>17</th>\n",
       "      <th>18</th>\n",
       "      <th>19</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3135</td>\n",
       "      <td>4719</td>\n",
       "      <td>1912</td>\n",
       "      <td>5076</td>\n",
       "      <td>149</td>\n",
       "      <td>269</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3731</td>\n",
       "      <td>5128</td>\n",
       "      <td>1889</td>\n",
       "      <td>2800</td>\n",
       "      <td>1026</td>\n",
       "      <td>266</td>\n",
       "      <td>5064</td>\n",
       "      <td>2228</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4039</td>\n",
       "      <td>4587</td>\n",
       "      <td>13</td>\n",
       "      <td>5076</td>\n",
       "      <td>77</td>\n",
       "      <td>6259</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>7450</td>\n",
       "      <td>136</td>\n",
       "      <td>7461</td>\n",
       "      <td>4753</td>\n",
       "      <td>7442</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6676</td>\n",
       "      <td>3844</td>\n",
       "      <td>4619</td>\n",
       "      <td>1534</td>\n",
       "      <td>251</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     0     1     2     3     4     5     6     7   8   9   10  11  12  13  14  \\\n",
       "0  3135  4719  1912  5076   149   269     0     0   0   0   0   0   0   0   0   \n",
       "1  3731  5128  1889  2800  1026   266  5064  2228   0   0   0   0   0   0   0   \n",
       "2  4039  4587    13  5076    77  6259     0     0   0   0   0   0   0   0   0   \n",
       "3  7450   136  7461  4753  7442     0     0     0   0   0   0   0   0   0   0   \n",
       "4  6676  3844  4619  1534   251     0     0     0   0   0   0   0   0   0   0   \n",
       "\n",
       "   15  16  17  18  19  \n",
       "0   0   0   0   0   0  \n",
       "1   0   0   0   0   0  \n",
       "2   0   0   0   0   0  \n",
       "3   0   0   0   0   0  \n",
       "4   0   0   0   0   0  "
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(X_train_padded_seqs).head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>10</th>\n",
       "      <th>11</th>\n",
       "      <th>12</th>\n",
       "      <th>13</th>\n",
       "      <th>14</th>\n",
       "      <th>15</th>\n",
       "      <th>16</th>\n",
       "      <th>17</th>\n",
       "      <th>18</th>\n",
       "      <th>19</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>7383</td>\n",
       "      <td>5263</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2155</td>\n",
       "      <td>5453</td>\n",
       "      <td>5076</td>\n",
       "      <td>149</td>\n",
       "      <td>5064</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4537</td>\n",
       "      <td>3946</td>\n",
       "      <td>12</td>\n",
       "      <td>5076</td>\n",
       "      <td>149</td>\n",
       "      <td>6259</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>6676</td>\n",
       "      <td>7237</td>\n",
       "      <td>162</td>\n",
       "      <td>1827</td>\n",
       "      <td>4933</td>\n",
       "      <td>2330</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>251</td>\n",
       "      <td>936</td>\n",
       "      <td>3501</td>\n",
       "      <td>251</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     0     1     2     3     4     5   6   7   8   9   10  11  12  13  14  15  \\\n",
       "0  7383  5263     0     0     0     0   0   0   0   0   0   0   0   0   0   0   \n",
       "1  2155  5453  5076   149  5064     0   0   0   0   0   0   0   0   0   0   0   \n",
       "2  4537  3946    12  5076   149  6259   0   0   0   0   0   0   0   0   0   0   \n",
       "3  6676  7237   162  1827  4933  2330   0   0   0   0   0   0   0   0   0   0   \n",
       "4   251   936  3501   251     0     0   0   0   0   0   0   0   0   0   0   0   \n",
       "\n",
       "   16  17  18  19  \n",
       "0   0   0   0   0  \n",
       "1   0   0   0   0  \n",
       "2   0   0   0   0  \n",
       "3   0   0   0   0  \n",
       "4   0   0   0   0  "
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(X_test_padded_seqs).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Convert (y) labels to vectors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Great',\n",
       " 'Good',\n",
       " 'Okay',\n",
       " 'Mediocre',\n",
       " 'Amazing',\n",
       " 'Bad',\n",
       " 'Awful',\n",
       " 'Painful',\n",
       " 'Unbearable',\n",
       " 'Masterpiece']"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "unique_y_labels = list(y_train.value_counts().index)\n",
    "unique_y_labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(unique_y_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LabelEncoder()"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn import preprocessing\n",
    "le = preprocessing.LabelEncoder()\n",
    "le.fit(unique_y_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# print('')\n",
    "# print(unique_y_labels)\n",
    "# print(le.transform(unique_y_labels))\n",
    "# print('')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "4: Great\n",
      "3: Good\n",
      "7: Okay\n",
      "6: Mediocre\n",
      "0: Amazing\n",
      "2: Bad\n",
      "1: Awful\n",
      "8: Painful\n",
      "9: Unbearable\n",
      "5: Masterpiece\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print('')\n",
    "for label_id, label_name in zip(le.transform(unique_y_labels), unique_y_labels):\n",
    "    print('%d: %s' % (label_id, label_name))\n",
    "print('')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "y_train = to_categorical(y_train.map(lambda x: le.transform([x])[0]), nb_classes=len(unique_y_labels))\n",
    "y_test  = to_categorical(y_test.map(lambda x:  le.transform([x])[0]), nb_classes=len(unique_y_labels))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.,  0.,  0.,  1.,  0.,  0.,  0.,  0.,  0.,  0.],\n",
       "       [ 0.,  0.,  0.,  0.,  1.,  0.,  0.,  0.,  0.,  0.],\n",
       "       [ 0.,  0.,  0.,  1.,  0.,  0.,  0.,  0.,  0.,  0.]])"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train[0:3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('y_train.shape:', (16759, 10))\n",
      "('y_test.shape:', (1863, 10))\n"
     ]
    }
   ],
   "source": [
    "print('y_train.shape:', y_train.shape)\n",
    "print('y_test.shape:', y_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Network Building"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "size_of_each_vector = X_train_padded_seqs.shape[1]\n",
    "vocab_size = len(vocab)\n",
    "no_of_unique_y_labels = len(unique_y_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('size_of_each_vector:', 20)\n",
      "('vocab_size:', 7599)\n",
      "('no_of_unique_y_labels:', 10)\n"
     ]
    }
   ],
   "source": [
    "print('size_of_each_vector:', size_of_each_vector)\n",
    "print('vocab_size:', vocab_size)\n",
    "print('no_of_unique_y_labels:', no_of_unique_y_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#sgd = tflearn.SGD(learning_rate=1e-4, lr_decay=0.96, decay_step=1000)\n",
    "\n",
    "net = tflearn.input_data([None, size_of_each_vector]) # The first element is the \"batch size\" which we set to \"None\"\n",
    "net = tflearn.embedding(net, input_dim=vocab_size, output_dim=128) # input_dim: vocabulary size\n",
    "net = tflearn.lstm(net, 128, dropout=0.6) # Set the dropout to 0.6\n",
    "net = tflearn.fully_connected(net, no_of_unique_y_labels, activation='softmax') # relu or softmax\n",
    "net = tflearn.regression(net, \n",
    "                         optimizer='adam',  # adam or ada or adagrad # sgd\n",
    "                         learning_rate=1e-4,\n",
    "                         loss='categorical_crossentropy')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Intialize the Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#model = tflearn.DNN(net, tensorboard_verbose=0, checkpoint_path='SavedModels/model.tfl.ckpt')\n",
    "model = tflearn.DNN(net, tensorboard_verbose=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train the Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training Step: 16799  | total loss: \u001b[1m\u001b[32m0.79447\u001b[0m\u001b[0m | time: 16.579s\n",
      "| Adam | epoch: 100 | loss: 0.79447 - acc: 0.7222 -- iter: 16700/16759\n",
      "Training Step: 16800  | total loss: \u001b[1m\u001b[32m0.78179\u001b[0m\u001b[0m | time: 17.683s\n",
      "| Adam | epoch: 100 | loss: 0.78179 - acc: 0.7270 | val_loss: 1.82861 - val_acc: 0.4348 -- iter: 16759/16759\n",
      "--\n"
     ]
    }
   ],
   "source": [
    "model.fit(X_train_padded_seqs, y_train, \n",
    "           validation_set=(X_test_padded_seqs, y_test), \n",
    "           n_epoch=n_epoch,\n",
    "           show_metric=True, \n",
    "           batch_size=100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Manually Save the Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:/media/amanullahtariq/Work/repo/github/MachineLearning/Challenge/TensorFlow/SentimentAnalysis/SavedModels/model.tfl is not in all_model_checkpoint_paths. Manually adding it.\n",
      "\u001b[31mModel Saved!\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "model.save('SavedModels/model.tfl')\n",
    "print(colored('Model Saved!', 'red'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Manually Load the Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from /media/amanullahtariq/Work/repo/github/MachineLearning/Challenge/TensorFlow/SentimentAnalysis/SavedModels/model.tfl\n",
      "\u001b[31mModel Loaded!\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "model.load('SavedModels/model.tfl')\n",
    "print(colored('Model Loaded!', 'red'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### RNN's Accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from sklearn import metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[36m\n",
      "RNN Classifier's Accuracy: 0.43478\n",
      "\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "pred_classes = [np.argmax(i) for i in model.predict(X_test_padded_seqs)]\n",
    "true_classes = [np.argmax(i) for i in y_test]\n",
    "\n",
    "print(colored('\\nRNN Classifier\\'s Accuracy: %0.5f\\n' % metrics.accuracy_score(true_classes, pred_classes), 'cyan'))\n",
    "#print(pred_classes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "-------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Show some predicted samples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Amy's Jigsaw Scrapbook Wireless Puzzle \n",
      "('pred_class:', 'Good')\n",
      "('true_class:', 'Good')\n",
      "\n",
      "DuckTales Remastered PlayStation 3 Platformer \n",
      "('pred_class:', 'Good')\n",
      "('true_class:', 'Good')\n",
      "\n",
      "NBA Live 08 PlayStation 3 Sports \n",
      "('pred_class:', 'Good')\n",
      "('true_class:', 'Okay')\n",
      "\n",
      "The Walking Dead: 400 Days PC Adventure, Episodic \n",
      "('pred_class:', 'Great')\n",
      "('true_class:', 'Great')\n",
      "\n",
      "Action Blox iPhone Action \n",
      "('pred_class:', 'Great')\n",
      "('true_class:', 'Okay')\n",
      "\n",
      "Kane & Lynch: Dead Men Xbox 360 Action \n",
      "('pred_class:', 'Good')\n",
      "('true_class:', 'Good')\n",
      "\n",
      "Life is Strange -- Episode 4: Dark Room Xbox One Adventure \n",
      "('pred_class:', 'Great')\n",
      "('true_class:', 'Okay')\n",
      "\n",
      "Genma Onimusha Xbox Action, Adventure \n",
      "('pred_class:', 'Okay')\n",
      "('true_class:', 'Great')\n",
      "\n",
      "Klonoa 2: Dream Champ Tournament Game Boy Advance Platformer \n",
      "('pred_class:', 'Okay')\n",
      "('true_class:', 'Great')\n",
      "\n",
      "The Walking Dead: A Telltale Game Series -- Season Two PlayStation Vita Adventure \n",
      "('pred_class:', 'Great')\n",
      "('true_class:', 'Great')\n",
      "\n",
      "Dead Star PlayStation 4 Strategy \n",
      "('pred_class:', 'Great')\n",
      "('true_class:', 'Good')\n",
      "\n",
      "Gods vs. Humans! Wii Strategy \n",
      "('pred_class:', 'Bad')\n",
      "('true_class:', 'Bad')\n",
      "\n",
      "Worms Revolution PlayStation 3 Strategy \n",
      "('pred_class:', 'Great')\n",
      "('true_class:', 'Great')\n",
      "\n",
      "I of the Dragon PC Action, RPG \n",
      "('pred_class:', 'Okay')\n",
      "('true_class:', 'Bad')\n",
      "\n",
      "Family Guy: Air Griffin Wireless Action \n",
      "('pred_class:', 'Bad')\n",
      "('true_class:', 'Bad')\n",
      "\n",
      "Tetris iPod Puzzle \n",
      "('pred_class:', 'Okay')\n",
      "('true_class:', 'Good')\n",
      "\n",
      "The Simpsons Wrestling PlayStation Action \n",
      "('pred_class:', 'Good')\n",
      "('true_class:', 'Unbearable')\n",
      "\n",
      "Godzilla PlayStation 4 Action \n",
      "('pred_class:', 'Painful')\n",
      "('true_class:', 'Bad')\n",
      "\n",
      "New World Order PC Shooter \n",
      "('pred_class:', 'Good')\n",
      "('true_class:', 'Bad')\n",
      "\n",
      "Blacklight: Retribution PC Shooter editors_choice\n",
      "('pred_class:', 'Amazing')\n",
      "('true_class:', 'Great')\n",
      "\n",
      "Sakura Wars: Hanagumi Taisen Columns 2 Dreamcast Puzzle \n",
      "('pred_class:', 'Great')\n",
      "('true_class:', 'Amazing')\n",
      "\n"
     ]
    }
   ],
   "source": [
    "ids_of_titles = range(0,21) # range(X_test.shape[0]) \n",
    "\n",
    "for i in ids_of_titles:\n",
    "    pred_class = np.argmax(model.predict([X_test_padded_seqs[i]]))\n",
    "    true_class = np.argmax(y_test[i])\n",
    "    \n",
    "    print(X_test.values[i])\n",
    "    print('pred_class:', le.inverse_transform(pred_class))\n",
    "    print('true_class:', le.inverse_transform(true_class))\n",
    "    print('')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "-----------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By Jovian Lin ([http://jovianlin.com](http://jovianlin.com))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
